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Abstract 

Three hundred learners participated in this study which (a) examined the instructional effectiveness with 
which different job aid types facilitated learner achievement of different types of learning objectives and 
(b) identified the degree to which low and high prior knowledge individuals profit from different job aid 
types. The instructional content used in the study was an instructional module on the parts and Junctions of 
the human heart. Participants were randomly assigned to one of five treatments complemented with 
different job aid types. After interacting with the instructional module, participants received four post tests 
each measuring different educational objectives. Research findings indicated that when job aids are used 
following instruction (a) they are not equally effective in facilitating learner achievement of different types 
of educational objectives; (b) they do not reduce learning differences between low and high prior 
knowledge learners; and (c) there is an insignificant interaction between job aid type and prior knowledge 
level. 

Introduction 

The essence of appropriate instruction relies heavily on an instructional design practitioner’s ability to 
balance instructional efficiency and effectiveness. Instructional efficiency takes into account finite resources such as 
time t money, and staff personnel allocated to the development of an instructional intervention while instructional 
effectiveness focuses on the desired performance or outcome of that intervention. To compound this situation in the 
corporate environment, performance demands change constantly because of the introduction of new products, 
services, process and technology; individuals are required to maintain current performance levels while acquiring 
new complex processes and skills; and management’s expectations have employees performing more tasks at 
irregular intervals. In meeting some of these diverse demands, instructional design practitioners have relied on non* 
instructional interventions known as job aids (Silber, 1990). Job aids, also known as performance support tools, 
contain factual and procedural knowledge and are used during actual task completion (Grau, 1986; Rossett, 1991; 

Roth well & Kazanas, 1994). They are effective especially when the consequence of error is high, performance is 
lengthy or complex, performance is changing frequently or limited budget or time exists for making an instructional 
intervention (Carlisle & Coulter, 1990; Finnegan, 1985). 

Duncan (1985, p.l) indicates that job aids “.. .put more training into the programs of instruction without 
significantly increasing the course length, ...save time and money in training development without sacrificing student 
achievement, . . .reduce the paperwork requirements in training development and multitude of training 
products. . .and. . .increase performance both initially and on a sustained basis.” Finnegan (1985) proposes the 
following five reasons to use any job aid type: (1) to provide a performance focus, (2) to guide performance that 
would likely be forgotten, (3) to reduce cost and development time — as compared to training, (4) to provide 
flexibility of revision when task change, and (5) to be used in conjunction with training to decrease training time. 
Evolving rapidly, job aids have taken on a multitude of shapes, sizes and forms. Utilizing a more formal approach to 
the development of job aids, Rakow (1981) suggests the following job aid typology; example, cueing, association, 
procedural ized, and analog classification. 

In recent years, the advancement of cognitive psychology has expanded the way individuals conceptualize 
job aids. Viewed in the past only as external mechanisms which store information and prompt memory to improve 
performance, job aids now have taken on additional rolls such as guiding perspectives, decisions and self evaluation 
(Rossett, 1991). Other individuals such as Tillman (1985) and Sleg (1988) suggest that job aids be incorporated as 
an instructional intervention to assist in the acquisition of skills and knowledge. Unfortunately, this paradigm shift 
has not occurred in the scientific community. Only a limited number of empirical research studies have been 
conducted using job aids. These studies have resulted in a contribution of valuable knowledge in the areas of: job 
aid and improved performance (Keefer, 1986; Lafleur, 1994); job aid development and implementation efforts 
(Ruyle, 1991; Ford, 1994); expertise transfer using knowledge based systems as job aids (Stevenson, 1990); and 
helpfulness of job aids in journal writing (Cole, 1993). Little research has been conducted to identify the potential 
educational value of job aids in the domain of knowledge acquisition. No research has been conducted to identify 
how prior knowledge interacts with job aids in the facilitation of knowledge acquisition. For this reason, an 
exploratory investigation was undertaken in this unknown area. 
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Statement of the Problem 



Literature supports the use of job aids to decrease efficiency (finite resources) and improve effectiveness 
(performance). But most literature indicates that job aids are used to recall, guide knowledge and 
procedures previously acquired. Limited research has been conducted using job aids as an instructional intervention. 
It was hypothesized that the use of job aids as an instructional intervention administered directly after interacting 
with the instructional content would function to facilitate the acquisition and retention of the content material. The 
study focused on the following research questions: 

• Are all types of job aids equally effective in facilitating learner achievement of different kinds of learning objectives? 

• Are different types of job aids equally effective in facilitating learner achievement among learners identified as possessing 
high or low prior knowledge? 

• is there an interaction between learners’ level of prior knowledge and type of job aid? 

• Do learners identified as high and low prior knowledge levels leam equally well from identical types of job aids? 



Procedure 

The physiology pretest was administered to nine hundred and three participants. Participation quartiles 
were calculated. Participants in the first quartile were identified as the low prior knowledge group; whereas 
participants in the fourth quartile were identified as the high prior knowledge group. Participants in the second and 
third quartiles were eliminated from the study. Each participant of the prior knowledge groups was then randomly 
assigned to one of five treatment subgroups. Total pretest participation quartiles are displayed in Table 1. 

Table L Total Pretest Participation Quartiles 



Percentage 


Quartile 


Pretest Score (Number Correct) 


100% 


Q4 


32-23 


75 % 


Q3 


22-21 


50% 


Q2 


20- 19 


25 % 


Ol 


18-03 



The cut off score for the low prior knowledge group was 18. Participants in each of the treatments 
possessing this score or lower were classified as possessing low prior knowledge. Combining all five treatments 
together the low prior knowledge group had a total of 247 participants. The high prior knowledge group for each 
treatment contained all participants who scored 23 or above on the pretest. Combining all five treatments together 
the high prior knowledge group had a total of 241 participants. The descriptive statistics for the total participation 
group, total low prior knowledge group and total high prior knowledge group are displayed in Table 2. 

Table 2. Total Participation Group, Total Low Prior Knowledge Group and Total High Prior Knowledge 
Group Descriptive Statistics 



Statistical Computation 


Total Participation 
Group 


Total Low Prior Knowledge 
Group 


Total High Prior Knowledge 
Group 


N 


903 


247 


241 


Mean 


20.05 


15.83 


25.25 


Standard Deviation 


4.80 


2.27 


2.06 


Variation 


23.00 


5.17 


4.25 



After identifying the low and high prior knowledge groups of each treatment, participants with incomplete 
data element were eliminated. Every seventh participant was then eliminated until all treatments contained the same 
number of participants and the prior knowledge groups (low and high) were distributed equally. The study was 
conducted with a sample of N=300. The low prior knowledge group consisted of 150 participants; high prior 
knowledge group consisted of 150 participants. Each of the five treatments was comprised of 60 participants with 30 
low prior knowledge and 30 high prior knowledge. Descriptive statistics for the sample based on low and high prior 
knowledge groups are shown in Table 3. 
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Table 3. Descriptive Statisticsjbr the Study Sample Based on Low and High Prior Knowledge 



Statistical Computation 


Low Prior Knowledge 


High Prior Knowledge 


N 


150 


150 


Mean 


15.62 


1 25.20 


Standard Deviation 


2.54 


2.07 


Minimum 


3 


23 


Maximum 


18 


32 



Treatments 

The instructional content used in both studies was a self-paced 1,800- word instructional module describing 
the human heart, its parts and the internal processes which occur during the systolic and diastolic phases (Dwyer, 
1972; Dwyer, 1978). This content was selected because of the high reliability associated with its dependent 
measures and because it offers the ability to investigate different intellectual skill learning objectives (facts, concepts, 
rules and problem solving) that are effective and efficient performance functions of a typical work environment. The 
job aid typology developed by Rakow (1981) provided the skeletal format for each job aid because each job aid type 
identified in his typology offers advantages when used correctly. In order words, specific job aid types must be 
developed for specific results. To improve learner performance, it is vital that the instructional design practitioner 
identifies the type of information required and how the information will be utilized (Gagne, Briggs & Wager; 1972). 
Keeping this in mind, each of the treatments developed for the study mirror a specific intellectual skills hierarchy 
level. The cueing job aid possessed facts; the association job aid possessed facts and concepts; the proceduralized 
job aid possessed facts, concepts, and rules/procedures; and the analog job aid possessed facts, concepts, 
rules/procedures and problem solving information. 

The same self-paced instructional module was distributed to all treatment groups. In addition to this 
module, all but the control group received some type of job aid and in some cases two job aids that were developed 
by the researchers after conducting an extensive literary search, collaborating with a content expert and determining 
reliability of the job aids through a pilot study. Job aids are thought to be self-sustaining, independent from training 
and for this reason, the researchers did not incorporate the job aids into the instructional content but placed them at 
the end of the instructional material to facilitate synthesization, integration and recall of the designated information. 
The five treatments described in Table 4 were developed specifically for the research study. 

Table 4. Treatment Descriptions 



Treatment 

Number 


Group 

Name 


Description 


1 


Control 


Participants received the instructional module containing no job aids. 


2 


Cueing 
Job Aid 


Participants received the instructional module containing two job aids designed 
to direct attention to the facts but did not provide step-by-step directions. The 
first job aid contained a heart graphic with structural parts labeled and the 
second 

job aid contained a heart graphic with labeled processes associated with blood 
circulation in the heart. The information in the cueing job aids provided 
participants with specific information they needed to discriminate between 
presented stimuli. 


3 


Association 
Job Aid 


Participants received the instructional module containing one job aid which 
related unknown information to already known information in the typical format 
of a reference document. The job aid contained facts using a heart graphic with 
labeled parts numbered and concepts in the form of a definition. The numbers 
identified the heart parts and their respective definitions. This job aid enabled 
the 

participant to identify a stimulus as a member of a group possessing some 
common characteristics, even though the stimuli would differ from each other. 


4 


Proceduralized 
Job Aid 


Participants received the instructional module containing one job aid describing 
the twelve step-by-step process of how the blood circulates through the heart 
Each step had an explanation and a heart graphic. The job aid used facts, 
concepts 

and a procedure associated with the third intellectual skill level. 


5 


Analog 
Job Aid 


Participants received the instructional module containing two job aids that 
assisted participants to conceptualize knowledge of organization, structure and 
relationships to solve problems and/or generate higher level concepts and values 
necessary to solve problems. One analog job aid presented knowledge of heart 
valve relationship; the other job aid provided knowledge of the circulation of 
blood through the heart. 
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Criterion Measures 



Four individual criterion measures, each measuring different learning objectives, were used as the cognitive 
dependent variables. The criterion measurers were administered immediately after the participants completed 
interacting with their respective self-paced instructional treatments. Table 5 summarizes the criterion measures. The 
listed descriptions are adapted from a description provided by Dwyer (1978, pp. 45-47) and illustrates the types of 
learning objectives that were assessed in this study. KR reliability coefficients are provided for each measure. 



Table 5. Description of Criterion Measures 



Criterion Measures 


Description 


Drawing Test 


Participants recalled information found in the instructional module to construct a 

simple line drawing of the heart. This 20-item test evaluated the participants’ ability to construct and/or 

reproduce content related items in their appropriate context (KR-20. 0.90). 


Identification Test 


Participants recalled information presented in the instructional module to identify 

parts and positions of an object. The test contained 20 multiple-choice questions (KR-20. 0.84). 


Terminology Test 


The objective of this 20-item, multiple-choice test was to evaluate the participants’ 
understanding of facts, terms and definitions printed in the instructional module (KR-20. 0.77). 


Comprehension Test 


Consisting of 20 multi pie -choice items, this test measured the participants’ deep 
understanding of concepts related to the functioning of the heart (KR-20. 0.77). 


Total Criterion Score 


The score is the sum of adding the correct number of responses of the four criterion measures. This 
score provided total understanding of the instructional module (KR-20, 0.94). 


Physiology Pretest 


This 36-item, multiple-choice test was developed to measure the participants’ prior 
knowledge level of human anatomy and physiology (KR-20. 0.92). 



Design 

The research design was a randomized post-test only design (Campbell & Stanley, 1966). Participant 
scores on the criterion measures (drawing, identification, terminology and comprehension tests) were identified as 
the dependent variable. The independent variable was job aid types (cueing, association, procedural ized and analog) 
and prior knowledge (low and high). The study design, a 5 X 2 crossed design, may be represented as RSIU (5 
levels of job aids and 2 levels of prior knowledge). Alpha was set at the .05 level. A MANOVA was conducted to 
determine the overall effects where significant F ratios were obtained. Table 6 provides a summary of pretest and 
dependent variable descriptive statistics. 

Table 6. Pretest and Dependent Variable Post Test Descriptive Statistics Correlation , Means and Standard 



Deviation ( N=300 ) 





Pretest 


Drawing 

Test 


Identification Test 


Terminology Test 


Comprehension Test 


Pretest 


1.00 


0.40 


0.38 


0.48 


0.37 


Drawing Test 




1.00 


0.80 


0.63 


0.61 


Identification Test 






1.00 


0.64 


0.67 


Terminology Test 








1.00 


0.74 


Comprehension 

Test 


1.00 


Mean 


20.41 


8.86 


12.62 


9.91 


9.76 


Stan Dev 


5.33 


5.31 


4.61 


3.68 


4.18 


Minimum 


3 


0 


2 


2 


1 


Maximum 


32 


20 


20 


20 


20 



Table 7 presents the mean achievement and standard deviation scores achieved by participants in each job 
aid by criterion measure. Where as Table 8 depicts mean achievement and standard deviation scores achieved by 
participants for low and high prior knowledge levels. 
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Table 7 . Mean Achievement and Standard Deviation Scores for Treatment 



Treatment ^ 

Mean 


Drawing 
Test Mean 


Identification 
Test Mean 


Terminology 
Test Mean 


Comprehension Test 
Mean 


jG dntrol 


4.333 


10.083 


8.900 


8.650 


Cueing 


9.916 


13.483 


9.666 


9.616 


Association 


10.433 


13.300 


10.550 


9.866 


Proceduralized 


10.033 


13.516 


10.933 


11.150 


Analog 


9.066 


12.716 


9.516 


9.533 



Table 8. Mean Achievement and Standard Deviation Scores for Prior Knowledge Level 



Prior Knowlejjge"'' 


Drawing 


Identification 


Terminology 


Comprehension 


^^-^Mean 


Test Mean 


Test Mean 


Test Mean 


Test Mean 


High 


10.920 


14.253 


11.546 


11.266 


Low 


6.793 


10.986 


8.280 


8.260 



The multivariate analyses (MANOVA) indicated that for all variables an insignificant interaction existed 
between job aid type and prior knowledge level. However, for all variables assessed simultaneously, a main effect 
existed among the different job aid treatments and between the prior knowledge levels. The results of the MANOVA 
are displayed in Table 9. 



Table 9. MANOVA Test Criteria and F Approximations for Overall Effect Using Wilks' Lambda Statistic 



Variable 


Value 1 


F | 


Num DF I 


Den DF 


Pr > F 


Job Aid 


0.767878466 


4.95215 


16 


877.437 


0.0001 


Prior Knowledge 


0.762896938 


22.2994 


4 


287.000 


0.0001 


Job Aid*Prior 
Knowledge 


0.931476308 


1.28913 


16 


877.437 


0.1965 



Follow up analyses indicated that all four job aid treatments were significantly more affective than was the 
control treatment on all four of the criterion measures. However, all job aids were found to be equally effective in 
facilitating learner achievement on each of the criterion measures. Follow up analyses also indicated that learners in 
the high prior knowledge level achieved significantly higher mean achievement scores on each criterion measure 
than did learners in the low prior knowledge level. 

Results 



The multivariate analyses (MANOVA) yielded an insignificant interaction between job aid type and level of 
prior knowledge (Table 9). As shown in Table 9 for all variables simultaneously, a main effect existed for job aids 
(F=4.95) and prior knowledge (F=22.23). 

Simultaneous confidence intervals using the .05 alpha level were calculated for each criterion measure to 
determine job aid treatment effects. On the drawing and identification measures all four job aid treatments (cueing, 
association, procedural ized, and analog) were found to be significantly more effective than the control in facilitating 
student achievement. Insignificant differences in achievement were found to exist among participant mean 
achievement scores on both criterion measures. On the terminology test measure, only treatments three (association) 
and four (proceduralized) were more effective than the control treatment in facilitating student achievement however, 
all job aid treatments were again found to be equally effective. On the comprehension test measure, only treatment 
four (proceduralized) was found to be significantly more effective than the control treatment in facilitating 
achievement. Insignificant differences in achievement were found to exist among the four job aid treatments. 

To identify the prior knowledge treatment effects, simultaneous confidence intervals were computed using 
the .05 alpha level. Results indicated that low prior knowledge learners achieved significantly lower mean 
achievement scores on all criterion tests (drawing, identification, terminology, and comprehension) than did high 
prior knowledge learners. 

Discussion 

The findings of this study indicated that job aid treatments (T2, T3, T4, T5) were all significantly more 
effective in facilitating learner achievement on the drawing and identification measures than was the control 
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treatment (Tl) which contained only the instructional module. A possible explanation for these results may be that 
all job aid treatments contained a graphical picture highlighting or identifying the structural and organizational 
components of the heart. These graphics helped the learners organize information to facilitate acquisition, retention 
and subsequent retrieval of the information. Research supports the fact that long term memory visualization 
increases performance (Kolers & Ostry, 1974; Light, Berger & Bardales, 1975). Gagne, Walker Yekovich and 
Yekovich (1993) suggest that declarative knowledge (knowing that something is the case) is stored in knowledge 
structures in the form of a schema. Schemas are characterized as having variables, being organized hierarchically 
(embedded within another), and facilitating inferences. Within schemas, declarative knowledge is represented in the 
form of a proposition, image or linear ordering. A proposition is a basic information unit such as one idea and 
expresses the relationships among concepts. According to Wanner (1968) individuals do not store knowledge in 
words, phrases or sentences but proposition form. The control group (Tl) participants receiving the instructional 
module containing only a narrative may have developed propositions without the assistance of job aids. However, 
participants who received the instructional module containing job aids may have developed propositions from the 
instructional narrative but also enriched the schema by adding additional declarative knowledge in the visual form of 
a heart graphic and linear ordering (step 1, step 2, step 3, etc.). 

On the terminology measure, only treatment three (association) and four (procedural ized) were more 
effective than the control treatment. The explanation for this finding may be that the association job aid required 
additional participant interactivity with the instructional modules containing job aids. Participants looked at the 
graphic, identified a number related to a specific heart part, identified the heart part, located the definition of the 
heart part listed at the bottom of the job aid below the graphic, read the definition and related the definition back to 
the heart part located on the graphic. Used as an elaboration learning strategy this activity influenced the 
construction and integration aspects of the encoding process. Internal connections are constructed in working 
memory between new information and knowledge stored in long term memory. The procedural ized job aid used an 
organizational strategy which not only identified and related major and minor information but also assisted in 
selection or the paying attention to incoming information which was then transferred into working memory and 
construction or the building of connections between ideas contained in working memory. Both the association and 
procedural ized job aids provided relationship information and required additional interaction with the instructional 
content. Braune and Forshay (1983) and Craik and Tulving (1975) both suggest that it is only through many 
interactions with the task that individuals seem to develop an understanding of functional relationships contained in 
concepts. The cueing job aids contained factual information and required very little interaction. The complexity of 
the analog job aids may have produced confusion or affected the ability of the participants to discriminate relevant 
information for non-relevant information. Non-processed information is not locatable and retrievable during the 
assessment phase and therefore, negatively impacts achievement. 

All job aids provided additional declarative knowledge in image form and may be the reason that all job 
aids did not differ significantly from one another. However, the additional interactivity of the association and 
procedural ized job aids was not enough to result in a significant difference between the job aid types. 

High and low prior knowledge participants did not profit equally from the different job aid treatments. 
Participants classified as low prior knowledge had significantly lower mean identification, drawing, terminology and 
comprehension scores than did high prior knowledge participants. This finding is consistent with the premise that 
successful integration of new information occurs when it is related to existing long term information (Bransford & 
Franks, 1976; Clifton & Slowiaczek, 1981; Davey & Kapinus, 1985). High prior knowledge participants who 
received the same treatments consistently scored higher on all four criterion post tests. According to Weinstein and 
Mayer (1986), if an individual already possess prior knowledge of a subject, it is easier to add information, connect a 
new schema to make inferences and access the schema information. 

All treatments contained in the study were developed as self-contained instructional books containing 
directions and instructional narrative and in some cases job aids. No additional assistance was rendered to the study 
participants. Therefore, the findings of the study may support Jonassen and Grabowski (1993) who found that high 
prior knowledge learners require less instructional assistance than do low prior knowledge learners and Willoughby 
and Wood (1994) who found that restriction of prior knowledge impacts the learners ability to integrate new 
information and makes instruction more difficult and increase the probability that achievement will be affected in a 
negative manner. 

Finally, the significant lower scores for low prior knowledge participants may be the result of their inability 
to make inferences (Gagne, Walker Yekovich & Yekovich, 1993). Experts possess more declarative knowledge and 
rich domain specific relevant information than do novices and possess the ability to make correct inferences during 
the spread of activation in long term memory when they are unable to retrieve exact information (Weinstein & 
Mayer, 1986). Novices, on the other hand, are less likely to make correct inferences because of limited knowledge 
representation in the cognitive domain. 
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Conclusion 



The research findings indicate that the type of paper-based job aid used with a self-paced instructional 
module would be based on the desired education objective. Also, if learners possess a high degree of prior 
knowledge, then paper-based job aids may be used with self-paced instructional content to positively effect the types 
of cognitive development evaluated in this study. The findings also indicate that to improve job aid effectiveness 
some form of interactivity such as rehearsal, elaboration, organizational or comprehension monitoring strategies be 
incorporated into the design of the job aids. 

The study was an attempt to venture into an uncharted area of using paper-based job aids as an instructional 
intervention to improve learner achievement. However, the findings do not provide enough evidence to alter the 
current practice of using job aids as non -instructional interventions. It is vital that additional research be conducted 
to substantiate the findings of this study and investigate additional variables which may impact on the use of job aids 
as instructional interventions. Future research may include but is not limited to the following list of potential 
empirical research: 



• Replicate the current study to substantiate findings and increase credibility. 

• Replicate the study substituting Factor B, prior knowledge, with time-on-task classifications to determine if the amount of 

time devoted to the instructional content and job aid alters the current achievement outcomes. 

• Replicate the study using a different instructional delivery medium. Use of a computer-based instructional module 
containing job aids may significantly impact the current achievement outcomes. 

• Replicate the study using job aids containing additional forms of learning strategies. 

• Replicate the study comparing the self-paced instruction with instructor-led instruction. 

• Conduct a longitudinal study to assess the effect of knowledge retention when paper-based job aids are administered after 
a self-paced instructional module. 
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